Application-specific Hardware Accelerators Research Articles

Integrated Photonic Tensor Processing Unit for a Matrix Multiply: A Review

The explosion of artificial intelligence and machine-learning algorithms, connected to the exponential growth of the exchanged data, is driving a search for novel application-specific hardware accelerators. Among the many, the photonics field appears to be in the perfect spotlight for this global data explosion, thanks to its almost infinite bandwidth capacity associated with limited energy consumption. In this review, we will overview the major advantages that photonics has over electronics for hardware accelerators, followed by a comparison between the major architectures implemented on Photonics Integrated Circuits (PIC) for both the linear and nonlinear parts of Neural Networks. By the end, we will highlight the main driving forces for the next generation of photonic accelerators, as well as the main limits that must be overcome.

Modular Hardware/Software Architecture for Edge Units in Highly Flexible Manufacturing Systems

With the increasing demand for individualized production and swift adaptability to changing market needs, new manufacturing system concepts emerge. Aiming for high flexibility and scalability, they rely on universal reconfigurable machines that can be used in various production steps by automatically changing tools and sensors. Consequently, the data processing infrastructure is required to support the frequent exchange of process-specific software applications, ranging from control and monitoring tasks to complex sensor data processing. This can be a major challenge when computation-intensive software is supported by hardware acceleration to achieve the desired performance or latency requirements. In this paper, we present an edge unit architecture that allows for hosting such applications while meeting the needs for modularity and reconfigurability. It builds upon a hypervisor to partition fixed processing resources among the applications. We present a mechanism that extends this partitioning to field-programmable gate arrays (FPGA) by using dynamic partial reconfiguration. This provides the option to deploy application-specific hardware accelerators with computation-intensive applications while maintaining modularity and on-line reconfigurability. We conclude the paper by pointing out the potential of the proposed architecture based on a use case for automated disassembly in a reconfigurable robotic cell.

Formal verification of high-level synthesis

High-level synthesis (HLS), which refers to the automatic compilation of software into hardware, is rapidly gaining popularity. In a world increasingly reliant on application-specific hardware accelerators, HLS promises hardware designs of comparable performance and energy efficiency to those coded by hand in a hardware description language such as Verilog, while maintaining the convenience and the rich ecosystem of software development. However, current HLS tools cannot always guarantee that the hardware designs they produce are equivalent to the software they were given, thus undermining any reasoning conducted at the software level. Furthermore, there is mounting evidence that existing HLS tools are quite unreliable, sometimes generating wrong hardware or crashing when given valid inputs. To address this problem, we present the first HLS tool that is mechanically verified to preserve the behaviour of its input software. Our tool, called Vericert, extends the CompCert verified C compiler with a new hardware-oriented intermediate language and a Verilog back end, and has been proven correct in Coq. Vericert supports most C constructs, including all integer operations, function calls, local arrays, structs, unions, and general control-flow statements. An evaluation on the PolyBench/C benchmark suite indicates that Vericert generates hardware that is around an order of magnitude slower (only around 2× slower in the absence of division) and about the same size as hardware generated by an existing, optimising (but unverified) HLS tool.

A Binary Translation Framework for Automated Hardware Generation

As applications move to the edge, efficiency in computing power and power/energy consumption is required. Heterogeneous computing promises to meet these requirements through application-specific hardware accelerators. Runtime adaptivity might be of paramount importance to realize the potential of hardware specialization, but further study is required on workload retargeting and offloading to reconfigurable hardware. This article presents our framework for the exploration of both offloading and hardware generation techniques. The framework is currently able to process instruction sequences from MicroBlaze, ARMv8, and riscv32imaf binaries, and to represent them as Control and Dataflow Graphs for transformation to implementations of hardware modules. We illustrate the framework's capabilities for identifying binary sequences for hardware translation with a set of 13 benchmarks.

Memory-Augmented Neural Networks on FPGA for Real-Time and Energy-Efficient Question Answering

Memory-augmented neural networks (MANNs) were introduced to handle long-term dependent data efficiently. MANNs have shown promising results in question answering (QA) tasks that require holding contexts for answering a given question. As demands for QA on edge devices have increased, the utilization of MANNs in resource-constrained environments has become important. To achieve fast and energy-efficient inference of MANNs, we can exploit application-specific hardware accelerators on field-programmable gate arrays (FPGAs). Although several accelerators for conventional deep neural networks have been designed, it is difficult to efficiently utilize the accelerators with MANNs due to different requirements. In addition, characteristics of QA tasks should be considered for further improving the efficiency of inference on the accelerators. To address the aforementioned issues, we propose an inference accelerator of MANNs on FPGA. To fully utilize the proposed accelerator, we introduce fast inference methods considering the features of QA tasks. To evaluate our proposed approach, we implemented the proposed architecture on an FPGA and measured the execution time and energy consumption for the bAbI data set. According to our thorough experiments, the proposed methods improved speed and energy efficiency of the inference of MANNs up to about 25.6 and 28.4 times, respectively, compared with those of CPU.

RegionSeeker: Automatically Identifying and Selecting Accelerators From Application Source Code

Embedded systems present stringent and often conflicting requirements. On the one side, the need for high performance within a tight energy budget favors inflexible Application Specific Integrated Circuit (ASIC) implementations; on the other side, a short time-to-market demands programmability. Hybrid architectures such as special-purpose customized processors represent an attractive solution, as they are programmable by software, but use dedicated hardware to accelerate parts of the computation. In such a scenario, the capability of automatically identifying the computation parts to be realized in hardware is highly desirable, in order to reduce design time and effort. This paper aims at advancing the state-of-the-art in this field. We recognize that subgraphs of control flow graphs having a single input control point and a single output control point, that we call regions , are good targets for the synthesis of application specific hardware accelerators. We therefore provide a method to identify them and an LLVM-based toolchain (named RegionSeeker) that, analyzing a software application, automatically selects its most profitable regions given an area constraint. Experimental evidence shows that the accelerators identified by RegionSeeker provide a speedup of up to $4.6\boldsymbol {\times }$ and, on average, approximately 30% higher speedup is achieved compared to state-of-the-art identification techniques.

An Instruction Set Architecture for Machine Learning

Machine Learning (ML) are a family of models for learning from the data to improve performance on a certain task. ML techniques, especially recent renewed neural networks (deep neural networks), have proven to be efficient for a broad range of applications. ML techniques are conventionally executed on general-purpose processors (such as CPU and GPGPU), which usually are not energy efficient, since they invest excessive hardware resources to flexibly support various workloads. Consequently, application-specific hardware accelerators have been proposed recently to improve energy efficiency. However, such accelerators were designed for a small set of ML techniques sharing similar computational patterns, and they adopt complex and informative instructions (control signals) directly corresponding to high-level functional blocks of an ML technique (such as layers in neural networks) or even an ML as a whole. Although straightforward and easy to implement for a limited set of similar ML techniques, the lack of agility in the instruction set prevents such accelerator designs from supporting a variety of different ML techniques with sufficient flexibility and efficiency. In this article, we first propose a novel domain-specific Instruction Set Architecture (ISA) for NN accelerators, called Cambricon, which is a load-store architecture that integrates scalar, vector, matrix, logical, data transfer, and control instructions, based on a comprehensive analysis of existing NN techniques. We then extend the application scope of Cambricon from NN to ML techniques. We also propose an assembly language, an assembler, and runtime to support programming with Cambricon, especially targeting large-scale ML problems. Our evaluation over a total of 16 representative yet distinct ML techniques have demonstrated that Cambricon exhibits strong descriptive capacity over a broad range of ML techniques and provides higher code density than general-purpose ISAs such as x86, MIPS, and GPGPU. Compared to the latest state-of-the-art NN accelerator design DaDianNao [7] (which can only accommodate three types of NN techniques), our Cambricon-based accelerator prototype implemented in TSMC 65nm technology incurs only negligible latency/power/area overheads, with a versatile coverage of 10 different NN benchmarks and 7 other ML benchmarks. Compared to the recent prevalent ML accelerator PuDianNao, our Cambricon-based accelerator is able to support all the ML techniques as well as the 10 NNs but with only approximate 5.1% performance loss.

Rapid Memory-Aware Selection of Hardware Accelerators in Programmable SoC Design

Programmable Systems-on-Chips (SoCs) are expected to incorporate a larger number of application-specific hardware accelerators with tightly integrated memories in order to meet stringent performance-power requirements of embedded systems. As data sharing between the accelerator memories and the processor is inevitable, it is of paramount importance that the selection of application segments for hardware acceleration must be undertaken such that the communication overhead of data transfers do not impede the advantages of the accelerators. In this paper, we propose a novel memory-aware selection algorithm that is based on an iterative approach to rapidly recommend a set of hardware accelerators that will provide high performance gain under varying area constraint. In order to significantly reduce the algorithm runtime while still guaranteeing near-optimal solutions, we propose a heuristic to estimate the penalties incurred when the processor accesses the accelerator memories. In each iteration of the proposed algorithm, a two-pass method is employed where a set of good hardware accelerator candidates is selected using a greedy approach in the first pass, and a “sliding window” approach is used in the second pass to refine the solution. The two-pass method is iteratively performed on a bounded set of candidate hardware accelerators to limit the search space and to avoid local maxima. In order to validate the benefits of the proposed selection algorithm, an exhaustive search algorithm is also developed. Experimental results using the popular CHStone benchmark suite show that the performance achieved by the accelerators recommended by the proposed algorithm closely matches the performance of the exhaustive algorithm, with close to 99% accuracy, while being orders of magnitude faster.

Guest Editors’ Introduction: Hardware Accelerators for Data Centers

It is our pleasure to introduce this special issue on hardware accelerators for data centers. Data centers around the world have been expanding and multiplying rapidly in the last decade with increased internet use, online services, compute consolidation, and data analytics. Hardware accelerators are increasingly important architectural components in the context of data center customization to achieve high performance and lower energy. Prominent companies have introduced FPGA/GPU-based platforms for data centers. For example, IBM’s Coherent Accelerator Processor Interface and Intel’s Quick-Assist Accelerator Abstraction Layer enable the integration of CPUs and FPGAs/GPUs through coherent shared memory. Microsoft built the configurable cloud platform for data centers and demonstrated significant performance improvements for different workloads. In addition to FPGAs and GPUs, application-specific hardware accelerators are being integrated into platforms for widely used workloads such as compression, cryptography, and pattern matching. Google’s tensor processing unit is reported to be used to accelerate machine learning (ML) workloads at Google’s data centers.

FPGAs: Fundamentals, Advanced Features, and Applications in Industrial Electronics [Book News]

This book presents all the facets of today’s field-programmable gate array (FPGA) technology. The aim of the authors is to offer engineers and, more generally, readers who have experience in industrial electronics a good understanding of both the basics and the latest trends related to FPGAs. This task is not easy because, since their creation 30 years ago, FPGAs have always been at the edge of innovation in the field of digital electronics, and, as a consequence, they are constantly evolving. This is all the more true because, over the last five years, a deep technological mutation has affected the world of FPGAs, shifting them from the well-known regular FPGA fabric to field-programmable systems on chips that, apart from the standard FPGA fabric, also include multiprocessor cores with their ecosystem. Thus, with the great diversity of possibilities offered by this new generation of devices, the risk to both the software engineer and the hardware designer is high if they are not able to find good documents that will help them demystify the part of the component with which they are not familiar. Having clearly identified this risk, the great merit of this book is to expose, in a simple way but with rigor, all the complexity and diversity of the latest generation of FPGAs. Readers will be able to better understand why such devices can fulfill the forthcoming challenges in terms of complexity for the next generation of industrial control systems. In today’s FPGAs, one can program at the same time powerful processor cores and design application-specific hardware accelerators in the FPGA fabric.

Cambricon

Neural Networks (NN) are a family of models for a broad range of emerging machine learning and pattern recondition applications. NN techniques are conventionally executed on general-purpose processors (such as CPU and GPGPU), which are usually not energy-efficient since they invest excessive hardware resources to flexibly support various workloads. Consequently, application-specific hardware accelerators for neural networks have been proposed recently to improve the energy-efficiency. However, such accelerators were designed for a small set of NN techniques sharing similar computational patterns, and they adopt complex and informative instructions (control signals) directly corresponding to high-level functional blocks of an NN (such as layers), or even an NN as a whole. Although straightforward and easy-to-implement for a limited set of similar NN techniques, the lack of agility in the instruction set prevents such accelerator designs from supporting a variety of different NN techniques with sufficient flexibility and efficiency. In this paper, we propose a novel domain-specific Instruction Set Architecture (ISA) for NN accelerators, called Cambricon, which is a load-store architecture that integrates scalar, vector, matrix, logical, data transfer, and control instructions, based on a comprehensive analysis of existing NN techniques. Our evaluation over a total of ten representative yet distinct NN techniques have demonstrated that Cambricon exhibits strong descriptive capacity over a broad range of NN techniques, and provides higher code density than general-purpose ISAs such as ×86, MIPS, and GPGPU. Compared to the latest state-of-the-art NN accelerator design DaDianNao [5] (which can only accommodate 3 types of NN techniques), our Cambricon-based accelerator prototype implemented in TSMC 65nm technology incurs only negligible latency/power/area overheads, with a versatile coverage of 10 different NN benchmarks.

Simultaneous accelerator parallelization and point-to-point interconnect insertion for bus-based embedded SoCs

As performance requirements for bus-based embedded System-on-Chips(So Cs) increase, more and more on-chip application-specific hardware accelerators(e.g., filters, FFTs, JPEG encoders, GSMs, and AES encoders) are being integrated into their designs. These accelerators require system-level tradeoffs among performance, area, and scalability. Accelerator parallelization and Point-to-Point(P2P) interconnect insertion are two effective system-level adjustments. The former helps to boost the computing performance at the cost of area,while the latter provides higher bandwidth at the cost of routability. What’s more, they interact with each other. This paper proposes a design flow to optimize accelerator parallelization and P2 P interconnect insertion simultaneously.To explore the huge optimization space, we develop an effective algorithm, whose goal is to reduce total So C latency under the constraints of So C area and total P2 P wire length. Experimental results show that the performance difference between our proposed algorithm and the optimal results is only 2.33% on average, while the running time of the algorithm is less than 17 s.

Multicore Signal Processing Platform With Heterogeneous Configurable Hardware Accelerators

The computing demand of many signal processing algorithms is dramatically growing because of the increasing complexity of embedded software applications. Concurrently, as process technology scales, the design effort for realizing very large scale integrated circuits and the associated costs are becoming critically high. A possible solution to address this performance/costs challenge is given by customizable multiprocessor system-on-chips. The approach proposed in this paper leads to the customization of multi/many processor system-on-chip at two levels of abstraction: 1) customization through application-specific hardware accelerators implemented on configurable datapath that can target three kinds of structured application-specific integrated circuit technologies: metal, via, and runtime programmable and 2) customization of the architectural parameters of the platform. The proposed platform is equipped with a design framework that assists the user in the high-level design-space exploration of signal processing applications described using the Open Computing Language (OpenCL) language. A peculiar added value of the flow is to support the migration of OpenCL kernels and tasks into pipelined hardware accelerators described using a C-level language. The platform is able to provide an average performance of 90 GOPS on a set of reference signal processing applications, and an average computational energy efficiency of 130 GOPS/W in its metal-programmable configuration. This result shows the benefits in terms of energy efficiency of hardware customization applied to multiprocessor systems with respect to many core devices such as general-purpose graphic processing units, able to provide on average 2.5 GOPS/W for the applications under analysis.

Protein Sequence Pattern Matching: Leveraging Application Specific Hardware Accelerators

Digitalization has brought a tremendous momentum to health care research. Recognition of patterns in proteins is crucial for identifying possible functions of newly discovered proteins, as well as analysis of known proteins for previously undetermined activity. In this paper, the workload consists of locating patterns from the PROSITE database in protein sequences. We optimize the pattern search task by using a new breed of processors that merge network and server attributes. We leverage massive multithreading and regular-expression (RegX) hardware accelerators; the latter were designed and built for an entirely different application - high-bandwidth deep-packet inspection. Our multithreading optimization achieves 18x improvement, but by harnessing a RegX accelerator we were able to further demonstrate a significant 392x improvement relative to software pattern matching. Moreover, performance per area and power consumption are improved by multiple orders of magnitude as well.

Hardware reuse in modern application-specific processors and accelerators

Effective exploitation of the application-specific parallel patterns and computation operations through their direct implementation in hardware is the base for construction of high-quality application-specific (re-) configurable application specific instruction set processors (ASIPs) and hardware accelerators for modern highly-demanding applications. Although it receives a lot of attention from the researchers and practitioners, a very important problem of hardware reuse in ASIP and accelerator synthesis is clearly underestimated and does not get enough attention in the published research. This paper is an effect of an industry and academic collaborative research. It analyses the problem of hardware sharing, shows its high practical relevance, as well as a big influence of hardware sharing on the major circuit and system parameters, and its importance for the multi-objective optimization and tradeoff exploitation. It also demonstrates that the state-of-the-art synthesis tools do not sufficiently address this problem and gives several guidelines related to enhancement of the hardware reuse.

Efficient memory management for hardware accelerated Java Virtual Machines

Application-specific hardware accelerators can significantly improve a system's performance. In a Java-based system, we then have to consider a hybrid architecture that consists of a Java Virtual Machine running on a general-purpose processor connected to the hardware accelerator. In such a hybrid architecture, data communication between the accelerator and the general-purpose processor can incur a significant cost, which may even annihilate the original performance improvement of adding the accelerator. A careful layout of the data in the memory structure is therefore of major importance to maintain the acceleration performance benefits. This article addresses the reduction of the communication cost in a distributed shared memory consisting of the main memory of the processor and the accelerator's local memory, which are unified in the Java heap. Since memory access times are highly nonuniform, a suitable allocation of objects in either main memory or the accelerator's local memory can significantly reduce the communication cost. We propose several techniques for finding the optimal location for each Java object's data, either statically through profiling or dynamically at runtime. We show how we can reduce communication cost by up to 86% for the SPECjvm and DaCapo benchmarks. We also show that the best strategy is application dependent and also depends on the relative cost of remote versus local accesses. For a relative cost higher than 10, a self-learning dynamic approach often results in the best performance.